HPC@UCLA: Introduction to the Hoffman2 Cluster

Raffaella D'Auria, PhD

Learning Outcomes

1) what is the Hoffman2 Cluster

2) what can the Hoffman2 Cluster do for you

3) navigating the documentation

4) getting an account on the Hoffman2 Cluster

5) connecting to the Hoffman2 Cluster via terminal and SSH

6) opening graphical applications on the Hoffman2 Cluster

7) unix command line 101

8) Getting work done on the Hoffman2 Cluster

What is the Hoffman2 Cluster

Nomenclature

Compute node:

• A compute node is a server (computer) with multiple computing cores that can execute as many simultaneous processes or threads as many cores it has

Computing core:

• A basic unit of computation within a CPU

Login node:

• A server designed to ensure connectivity between the "world" and the computing nodes on the cluster

Nomenclature (cont'd)

Computing Cluster:

• A computing cluster is made of individual nodes (computers) which are interconnected between themselves and which aggregate power can be harnessed to address problems which nature requires either distributed or capacity computing

Learning Outcomes 2

1) what is the Hoffman2 Cluster

2) **what can the Hoffman2 Cluster do for you**

3) navigating the documentation

4) getting an account on the Hoffman2 Cluster

5) connecting to the Hoffman2 Cluster via terminal and SSH

6) opening graphical applications on the Hoffman2 Cluster

7) unix command line 101

8) Getting work done on the Hoffman2 Cluster

What can the Hoffman2 Cluster do for you

High Performance Computing

• distributed computing (multiple computing cores across multiple compute nodes)
• shared-memory computing (multiple cores on the same compute node)
• data-intensive computing
• high-memory computing

Capacity Computing

• multiple computing cores available to carry out very many simulations
• execute concurrent (synchronous) or indeopendent (asynchronous) simulations

What can the Hoffman2 Cluster do for you (cont'd)

Licensed software availability

• Abaqus
• Intel Parallel Cluster Studio
• Maple
• Mathematica
• MATLAB

Licensed software management

• bring your own licensed software (COMSOL, Ansys, FDTD, etc.)

User Support/Project Collaboration/Training

Learning Outcomes 3

1) what is the Hoffman2 Cluster

2) what can the Hoffman2 Cluster do for you

3) **navigating the documentation**

4) getting an account on the Hoffman2 Cluster

5) connecting to the Hoffman2 Cluster via terminal and SSH

6) opening graphical applications on the Hoffman2 Cluster

7) unix command line 101

8) Getting work done on the Hoffman2 Cluster

Navigating the documentation

https://www.hoffman2.idre.ucla.edu/

Learning Outcomes 4

1) what is the Hoffman2 Cluster

2) what can the Hoffman2 Cluster do for you

3) navigating the documentation

4) **getting an account on the Hoffman2 Cluster**

5) connecting to the Hoffman2 Cluster via terminal and SSH

6) opening graphical applications on the Hoffman2 Cluster

7) unix command line 101

8) Getting work done on the Hoffman2 Cluster

Getting an account on the Hoffman2 Cluster

• Any perspective Hoffman2 Cluster user needs to be approved by a Hoffman2 Cluster Faculty Sponsor
• UCLA faculty or researchers that qualifies under UCLA Policy 900 as principal investigators can apply to become Hoffman2 Cluster Faculty Sponsors
• Hoffman2 Cluster Faculty Sponsors can sponsor themselves and their collaborators to become Hoffman2 Cluster users
• All accounts matters are handled through the System Identity Manager (SIM)
• Access to SIM is via your UCLA Logon ID

More information are available at: https://www.hoffman2.idre.ucla.edu/Accounts/Accounts.html#

In what follow a demonstration of a registration as a H2C Faculty Sponsor and a H2C user will be show.

Connecting to SIM for the first time

• Access to the System Identity Manager (SIM) requires authentication via your UCLA Logon ID

• the first time you connect you should see:

• select NO if you do not have an account already

Registering with SIM as a Hoffman2 Faculty Sponsor

UCLA faculty or researchers that qualifies under UCLA Policy 900 as principal investigators can apply to become Hoffman2 Cluster Faculty Sponsors

Below is a screenshot with the information provided to the SIM registration page by the hypotetical PI, Éuro Bruin, who is applying to become a Hoffman2 Cluster Faculty Sponsor:

Registering with SIM as a Hoffman2 Faculty Sponsor (cont'd)

Sponsors registration requires the creation of a unix group, all perspective users applying under a PI sponsorship will be part of this unix group.

Registering with SIM as a Hoffman2 Faculty Sponsor - Approval Step

All Sponsors applications have to be verified and approved, this process generally take about a working day. After selecting the unix group name SIM will display:

Requesting a user account (for Sponsors or Sponsorees)

SIM currently does not send email notification, Sponsors need to log back into SIM to check the status of their application

Sponsors can sponsors themselves for a user account

After approval Sponsors can create an account for themselves and approve/deny new users requesting their sponsorship:

Requesting a user account (for Sponsors or Sponsorees) -cont'd

All perspective users, after registration with SIM can request a user account, the first step is to select a Hoffman2 Facutly Sponsor:

Completing the user cluster account request

Sponsors account approval or denial

After requesting an account perspective users should remind their selected sponsors to log into SIM and approve or deny their application.

SIM does not send email messages, if the sponsor does not log into SIM accounts requests will not be approved

This is what a PI who has pending account requests would see upon logging onto SIM:

Setting the password for your account

Upon being approved, users will need to log back into SIM (you will not be notified by SIM that your account has been approved, it is your responsibility to periodically check).

An approved user will see the following menu requesting to set a password for their newly created accounts:

Password resetting

Use the link: https://sim.idre.ucla.edu/sim/account/view and click on the link: Change the password for <YOURHOFFMAN2USERNAME> on the H2 cluster shown below for user dbruin.

Managing your account

https://www.hoffman2.idre.ucla.edu/Accounts/Users-managing-your-account.html#password-processes

Sponsor responsibilities

https://www.hoffman2.idre.ucla.edu/Accounts/Sponsors-managing-your-group.html#sponsor-responsibilities

For sponsor: managing users in your group

https://www.hoffman2.idre.ucla.edu/Accounts/Sponsors-managing-your-group.html#managing-your-users

Learning Outcomes 5

1) what is the Hoffman2 Cluster

2) what can the Hoffman2 Cluster do for you

3) navigating the documentation

4) getting an account on the Hoffman2 Cluster

5) **connecting to the Hoffman2 Cluster via terminal and SSH**

6) opening graphical applications on the Hoffman2 Cluster

7) unix command line 101

8) Getting work done on the Hoffman2 Cluster

Connecting to the Hoffman2 Cluster via terminal and SSH

https://www.hoffman2.idre.ucla.edu/Using-H2/Connecting/Connecting.html

Connecting via a terminal and an SSH client (from Linux)

https://www.hoffman2.idre.ucla.edu/Using-H2/Connecting/Connecting.html#connecting-via-terminal-and-ssh

Connecting via a terminal and an SSH client (from Mac)

https://www.hoffman2.idre.ucla.edu/Using-H2/Connecting/Connecting.html#connecting-via-terminal-and-ssh

Connecting via a terminal and an SSH client (from Windows)

https://www.hoffman2.idre.ucla.edu/Using-H2/Connecting/Connecting.html#connecting-via-terminal-and-ssh

Connecting via terminal and SSH - Linux, Mac & Windows

on a Windows PowerShell make sure that the version is 5.1 or later with the command:

 $PSVersionTable.PSVersion

on any terminal applications you may run (aside from Putty and Cygwin excluded in this presentation), type

 ssh <YOURHOFFMAN2CLUSTERUSERNAME>@hoffman2.idre.ucla.edu

Connecting via a terminal and SSH for the first time

https://www.hoffman2.idre.ucla.edu/About/FAQ/FAQ.html#connecting-for-the-first-time

Connecting for the first time hostkey fingerprints

https://www.hoffman2.idre.ucla.edu/About/FAQ/FAQ.html#table-h2-public-host-classes

Connecting via terminal and SSH (cont'd)

Set up SSH keys to connect to the Hoffman2 Cluster

SSH keys are authentication credentials that can be set up for passwordless authentication (use of a passphrase is strongly recommended).

On a terminal on your local computer generate SSH-key:

 ssh-keygen -f $HOME/.ssh/hoffman2.idre.ucla.edu

On a terminal on your local computer create a $HOME/.ssh/config using, for example, the nano editor:

 nano $HOME/.ssh/config

• NOTE: on a PowerShell terminal use instead:

 bash -c 'nano $HOME/.ssh/config'

On a terminal on your local computer add these lines to your $HOME/.ssh/config:

Host hoffman2
 hostname hoffman2.idre.ucla.edu
 ServerAliveInterval 30
 ServerAliveCountMax 5
 IPQoS throughput
 IdentityFile=~/.ssh/hoffman2.idre.ucla.edu

Set up SSH keys to connect to the Hoffman2 Cluster (cont'd)

copy the key to the Hoffman2 Cluster

 $ ssh-copy-id -i ~/.ssh/hoffman2.idre.ucla.edu <YOURH2CUSERNAME>@hoffman2.idre.ucla.edu

connect to the Hoffman2 Cluster

 $ ssh <YOURHOFFMAN2CLUSTERUSERNAME>@hoffman2

Learning Outcomes 6

1) what is the Hoffman2 Cluster

2) what can the Hoffman2 Cluster do for you

3) navigating the documentation

4) getting an account on the Hoffman2 Cluster

5) connecting to the Hoffman2 Cluster via terminal and SSH

6) **opening graphical applications on the Hoffman2 Cluster**

7) unix command line 101

8) Getting work done on the Hoffman2 Cluster

Opening graphical applications on the Hoffman2 Cluster

X Window Sytems/X11: The software that allow displaying of graphical applications on a unix-like system and that is designed to work on a server-client mode (you can connect to it and forward graphical applications to your local computer)

Example: forward the MATLAB desktop GUI from Hoffman2 to your local computer

Opening graphical applications on Linux (Hands-on)

• X11 server is natively available
• forwarding of graphcail applications for remote display is enabled via:

  ssh -X joebruin@hoffman2.idre.ucla.edu

  substitute joebruin with your Hoffman2 username

Opening graphical applications on MacOS (Hands-on)

first time only:

• install Xquartz (the X11 server for MacOS)
• open the terminal app and before connecting to Hoffman2 issue:

defaults write org.macosforge.xquartz.X11 enable_iglx -bool true

• reboot your local computer

connect with:

• forwarding of graphcail applications for remote display is enabled via:

  ssh -Y joebruin@hoffman2.idre.ucla.edu

  substitute joebruin with your Hoffman2 username

Opening graphical applications on Windows

no need to install other software then any of the following:

• MobaXterm
• Windows Subsystem for Linux
• Cygwin - NOT RECOMMENDED UNLESS YOU ARE AN EXPERT

need to install an X11 window server if using either of:

• GitBash
• PuTTY
• PowerShell

a possible X11 Window server:

• Xming

Opening graphical applications on Windows using MobaXterm (Hands-on)

Opening graphical applications on Windows using MobaXterm (cont'd)

Opening graphical applications on Windows using MobaXterm (cont'd)

Opening graphical applications on Windows using MobaXterm (cont'd)

Opening graphical applications on Windows using MobaXterm

Learning Outcomes 7

1) what is the Hoffman2 Cluster

2) what can the Hoffman2 Cluster do for you

3) navigating the documentation

4) getting an account on the Hoffman2 Cluster

5) connecting to the Hoffman2 Cluster via terminal and SSH

6) opening graphical applications on the Hoffman2 Cluster

7) **unix command line 101**

8) Getting work done on the Hoffman2 Cluster

Unix command line 101

https://www.hoffman2.idre.ucla.edu/Using-H2/Command-line/Unix-command-line.html

Unix command line 101: Navigation (Hands-on)

https://www.hoffman2.idre.ucla.edu/Using-H2/Command-line/Unix-command-line.html#navigation

• Try these commands:

ls
ls -a
ls -l
ls -lat
ls -latr
pwd
cd $SCRATCH
pwd
timestamp=`date "+%F_%H-%M"`; mkdir $HOME/H2HH_$timestamp; cd $HOME/H2HH_$timestamp
pwd
cd
rmdir $HOME/H2HH_$timestamp
ls $HOME/H2HH_$timestamp

Move to Hands-on notebook

Unix command line 101: Environmental Variables (Hands-on)

https://www.hoffman2.idre.ucla.edu/Using-H2/Command-line/Unix-command-line.html#unix-environment-environmental-variables

• Some important environmental variables:

$PATH
$LD_LIBRARY_PATH
$HOME
$SCRATCH  # Hoffman2-specific
$SHELL

• to check the content of an environmental variable issue:

echo $HOME

Move to Hands-on notebook

Unix command line 101: Working with files (Hands-on)

https://www.hoffman2.idre.ucla.edu/Using-H2/Command-line/Unix-command-line.html#working-with-files

• Try these commands:

  cat $HOME/.bashrc
  less $HOME/.bashrc   # type q to interrupt
  more $HOME/.bashrc   # type q to interrupt

• Check the first few lines of a file:

  head $HOME/.bashrc
  head -n 2 $HOME/.bashrc

• Check the last few lines of a file:

  tail $HOME/.bashrc
  tail -n 2 $HOME/.bashrc

• Check the last lines of a file as it is being written:

  tail -f $SGE_ROOT/$SGE_CELL/common/accounting   # Control-C to interrupt

Move to Hands-on notebook

Unix command line 101: Editing files (Hands-on)

https://www.hoffman2.idre.ucla.edu/Using-H2/Command-line/Unix-command-line.html#editing-files

• Non graphical editors:

  nano 
  emacs
  vi

• Graphical editors:

  gedit &
  emacs &
  gvim &

Learning Outcomes 8

1) what is the Hoffman2 Cluster

2) what can the Hoffman2 Cluster do for you

3) navigating the documentation

4) getting an account on the Hoffman2 Cluster

5) connecting to the Hoffman2 Cluster via terminal and SSH

6) opening graphical applications on the Hoffman2 Cluster

7) unix command line 101

8) **Getting work done on the Hoffman2 Cluster**

Getting work done on the cluster

On which resources will your job run?

https://www.hoffman2.idre.ucla.edu/Using-H2/Computing/Computing.html#computational-resources-on-the-hoffman2-cluster

Highp vs shared vs campus resources:

https://www.hoffman2.idre.ucla.edu/Using-H2/Computing/Computing.html#highp-vs-shared-vs-campus-jobs

• highp refers to the use of group owned compute nodes
  • users can run jobs for up to 14 days
  • only for users in group who own resources
• shared refers to the use of temporarely unused group owned compute nodes
  • users can run jobs for up to 24 hours
  • only for users in group who own resources
• campus refers to compute nodes owned by IDRE and made available to the UCLA community
  • users can run jobs for up to 24 hours

Do I have access to highp resources?

To find out paste in a terminal connected to the clster the command (omitting the $ character indicative of the unix prompt):

$ myresources

what do you see?

What computational resources do I have access to?

Open a terminal on the Hoffman2 Cluster and issue:

$ myresources

if the first line of your ouput contains:

User joebruin is in the following resource group(s): campus

you do NOT have access to group-owned compute nodes and can only run for up to 24 hours on nodes owned by OARC/IDRE

if the first line of your output contains:

User joebruin is in the following resource group(s): gobruins evebruin

you have access to the nodes purchased by groups: gobruins and evebruin and you can run for up to 24 hours on shared queues and for up to 14 days when requesting to run on owned resources (highp mode)

Working interactively on the Hoffman2 Cluster

More on this on future presentations

Any work that will use substantial computational resources should be run on compute nodes and not on the login nodes.

To get an interactive session on one core of a compute node, from a terminal issue the following command(omitting the $ character indicative of the unix prompt):

$ qrsh

What happens?

(To terminate your interactive session, after the prompt returns, type: Control + d or logout)

What applications/software is already available on Hoffman2?

Refer to: https://www.hoffman2.idre.ucla.edu/Using-H2/Software/Software.html

Apps available via modules

To see what applications are available in the current hierarchy, at a terminal connected to Hoffman2 issue the command:

$ module av # press enter to scroll down and exit the view

To look for a specific software, for example R, issue the command:

$ modules_lookup -m R

Worked out example of python environment creation

• log onto the Hoffman2 Cluster via termina and SSH, at the cluster command prompt issue:

 qrsh 
 module av python
 module av anaconda
 module av mamba

• in this example we will use the default python version 3.9.6:

module load python/3.9.6
mkdir -p $HOME/.venv/3.9.6
python -m venv $HOME/.venv/3.9.6/NEWENV
source $HOME/.venv/3.9.6/NEWENV/bin/activate
# now install any needed python package, for example:
pip install numpy
# for example, update pip:
python -m pip install --upgrade pip